Embodiments of the inventive concept described herein relate to an image stitching system and a method therefor, and more particularly, relate to technologies of stitching images using viewpoint transformation and technologies of transforming viewpoints of the images.
To stitch several images into one image to produce a panorama image is performed through a procedure of aligning images including the same subject (images, each of which has an overlapped portion), determining relative motion between the images, projecting the images onto a cylinder or sphere, and blending portions where the images are overlaid with each other, and has been studied for a long time in the computer vision field.
In detail, a manner of aligning images may include a manner to compare the images for each patch and obtaining optical flow and a manner of extracting feature points from an image and comparing the feature points. Particularly, the feature-based manner shows a more robust result with respect motion of a scene and has a faster processing speed. Furthermore, an adjacent relationship may be automatically identified from blended images to produce a panorama. Such a manner is disclosed in the paper, Brown, M. and Lowe, D., “Recognizing panoramas”, ICCV 2003, pp. 1218-1225, Nice Prance.
Furthermore, when registering and aligning images and determining motion, models such as two-dimensional (2D) transform, a planar perspective model, or three-dimensional (3D) camera rotation are used. Images may be overlaid and shown (i.e., the “ghost” may occur) due to parallax or motion of a subject in a joint or there may be a brightness difference between the images when stitching the images, so a manner of accurately blending two images, a joint with small texture of which is stitched or joined.
As such, stitching the images results in a ghost phenomenon where an image joint is blurry or overlapped, when there is parallax between the images. For example, in case of a 360-degree camera system configured with a plurality of cameras which are currently released and is widely used, there is parallax between images captured by the plurality of cameras and a seam is not matched in a joint of the images when the images are stitched. However, to capture images such that there is no parallax between the images, the images should be captured while a camera rotates physically with respect to an optical center of the camera. Since the optical center is located inside or in front of a camera lens system, when a camera system including two or more cameras captures images, there is always parallax since it is impossible to match optical centers of the cameras.
To match optical centers between images captured by several cameras, a manner of using a separate optical device such as a mirror is proposed. However, the manner has a disadvantage in which a viewing angle is limited and a disadvantage in which the bulk of a camera is increased. Such a manner is disclosed in the paper, Kar-Han Tan, Hong Hua, and Narendra Ahuja, “Multiview Panoramic Cameras Using Mirror Pyramids”, IEEE Trans. Pattern Analysis and Machine Intelligence 26:7 (2004), 941-945.
Moreover, a manner to partially modify each of images captured by several cameras is proposed. However, the manner is to minimize a mismatch in seam rather than to remove parallax and has a problem in which a partial distortion occurs in images.
Technologies of matching viewpoints of images to remove parallax which exists between the images, without a separate device while a camera does not rotate or move physically and technologies of synthesizing and stitching images, each of which has parallax, based on the corresponding technologies are proposed.